System and method for simultaneous display of multiple information sources

ABSTRACT

A computerized method of presenting information from a variety of sources on a display device. Specifically the present invention describes a graphical user interface for organizing the simultaneous display of information from a multitude of information sources. In particular, the present invention comprises a graphical user interface which organizes content from a variety of information sources into a grid of tiles, each of which can refresh its content independently of the others. The grid functionality manages the refresh rates of the multiple information sources. According to one embodiment, the method of the present invention allocates refresh rates to tiles according to priorities that are assigned based on identifiers such as quality of service (QoS) tags associated with one or more of the information sources. The present invention is intended to operate in a platform independent manner.

[0001] This application is a continuation-in-part of application Ser.No. 09/702,325 filed Oct. 30, 2000 to which it claims priority. Thisapplication also claims benefit of priority to provisional applicationSerial No. 60/162,522, filed Oct. 29, 1999.

FIELD OF THE INVENTION

[0002] The present invention relates to methods of presentinginformation from a variety of sources on a display device. Specificallythe present invention describes a graphical user interface fororganizing simultaneous display of information from a multitude ofsources.

BACKGROUND OF THE INVENTION

[0003] The scope of the global communications capacity, comprising fixedlink and wireless networks, continues to expand rapidly. The variety andcomplexity of communication devices proliferates and the number of usersescalates. As a result, users are faced with increasingly complexsystems and interfaces with which to manage multiple sources ofinformation. At the same time, society has increased its demands on timeand productivity so that users no longer have the luxury of focusingtheir attention on a single source of information or means ofcommunication. Instead, the norm today is for people to carry out manytasks simultaneously.

[0004] As might be expected, these demands have exposed substantialproblems in current communications technology. In particular, users arefaced with insufficient resources to manage and access the volume andvariety of information available to them in an efficient and productivemanner. While a variety of tools designed to assist in accessing andmanaging these resources have been created, these tools remainunsatisfactory. Consequently, users are impeded by the myriad ofinformation sources, each with its own method of use and often with itsown login and password requirements, as well as by slow retrieval timesto access the information. The result is an unacceptable delay for manyoperations.

[0005] Under the present art, for example, it is usually the case that auser lacks the bandwidth resources to receive multiple video signalssimultaneously. If an individual were receiving one video signal, it isusually impractical to receive a second at the same time due tobandwidth constraints. Thus, the user could not, for example, monitormultiple video data streams of sporting or news events; instead, theuser could monitor only one video data stream at a time.

[0006] To address such bandwidth resource limitations, the current artonly accesses information when the user requests it. As a result, thereis an inevitable delay between the user's request for information andthe communications device's presentation of it. For example, if a userwants to monitor sources of news information on the Internet usingcurrent browser technology, the user must continuously and manuallyrequest the news data from its source to determine whether the data hasbeen updated. Prior to requesting and subsequently receiving the data,the user has no way of knowing whether the data has been updated. In anycase, the user is unlikely to want to refresh the status of eachapplication by manual intervention himself at the frequency necessary toensure that the information is up to date. Additionally, if a userwishes to view two or more webpages simultaneously, he must run two ormore copies of the web-browser program. The act of manually refreshingthe content of alternate programs in order to ascertain which have anynew material to offer is fundamentally inefficient.

[0007] Similarly, the user's access to such data is not in real-time oreven near real-time because each time the user wants to view theinformation, he must request it from its source and wait for the sourceto transmit it to him. Thereafter, he must wait until his communicationsdevice has received and processed the information before it ispresented. For complex information such as a video signal, this can takelonger than a minute to occur; and, even for simple information, thisprocess can take many seconds. Thus, the user is denied real-time ornear real-time access to the information.

[0008] Present technology that locally stores or “caches” previouslyaccessed information to make it available to the user more rapidly doesnot solve this problem, because the cached information is necessarilyold. The user's communications device must still verify the accuracy ofthe information with the source before the system displays the cachedinformation. As a result, the user is denied real-time or near real-timeaccess to updated information.

[0009] Similarly, if a user wishes to make two or more simultaneousdownloads there is no control over the relative rates at which therespective downloads would occur. So-called “push technologies” attemptto address this problem by organizing information from a number ofrelated sources and sending it periodically to a user. While thisarrangement frees a user from actively participating in the download,the price is that the user has little control over the organization ofthe information and can only practically handle a small number of suchtransmissions at any one time. Each transmission is subject to thebandwidth available.

[0010] Of course, not all tasks require the same allocation of resourcesand, correspondingly, not all tasks have equal priorities for a givenuser. In particular, a user may wish to customize the informationenvironment in such a way that many processes are occurringsynchronously, yet each is communicating with the user at a rate that isacceptable. For example, a television viewer may wish to know what isbeing broadcast on several channels at the same time but only care towatch one of them closely. An Internet user may wish to be continuallyin touch with sources of data from audio, video, chat-room,video-conferencing and e-mail checker utilities, but not wish all ofthem to update at the same frequency; the user would be satisfied merelyto see at a glance a recent status of each. Some of these processes,such as chat-room activities entail very little data transmission andcan, indeed, be effectively updated on a continuous basis, whereasothers require a great deal of bandwidth but could usefully be sampledat a lower rate. The current art lacks any technology for controllingthe respective refresh rates of several simultaneous informationsources. Furthermore, the current art lacks technology that canautomatically design an appropriate refresh rate based on the type ofdata that is received.

[0011] At the same time that users are limited by system resources, theyare also finding that they have no effective way of managing themultiplicity of available data types and information sources. It isdifficult both to conduct two or more different types of computingactivities at the same time or to monitor two or more differentinformation sources simultaneously because the tools available areconfusing, inflexible, and/or otherwise difficult to implement. Usersrequire immediate access to a wide variety of up to date contentpresented in a flexible, easily customized interface.

[0012] In addition to restrictions in the capacity of today's networks,there is very little conformity amongst the information content. Atypical communication device, such as a personal computer, television ormobile telephone, comprises a display unit connected to a processingunit that can accept information from many different sources. Asdescribed above, the signals, data and/or datastreams that are availableto such a device are diverse, including, for example, HTML content,e-mail, or streaming audio and video. Correspondingly, the softwaretools that interpret and process the different information sourcespresent each in a different way to the user. From a user's perspective,distinctions between the different types of information could usefullybe removed so that each is viewed in a similar way and such that thecurrent presentation associated with any information source gives animmediate indication of its current content. The present reality isdifferent, however. The user must contend with a wide range of icons andprogram windows that may occupy space on a user's display screen.Another lack of conformity is the different mode of behavior forprograms that address different types of information. An effort tostandardize the ways in which different types of information arepresented to the user would be advantageous. Equally, unification of theway in which those types of information are managed would save time andincrease user productivity, for productivity is reduced when users mustcope with different attributes of different programs and learn distinctparadigms for different types of information.

[0013] The nature of the application program windows and theirrespective icons predominantly found on today's computer displays isrestrictive. The application window typically displays the currentcontent or output of only a single program and program icons conveynothing of the program's current state or content. Often, an icon is astatic image which is merely characteristic of the program or datarepresented thereby rather than the program's current state or itsinformation content. In the present art, there is no intermediatebetween a window or an icon.

[0014] Thus, while a window may be resized as appropriate, it willfrequently occupy the full display area, effectively limiting the userto a view of a single program. It may have active areas around itsborders such as menu bars, scroll bars, or tool bars designed to allowthe user to control aspects of the window's appearance or to setparameters specific to the operation of the program controlling it.Icons, in contrast, offer ease of display when multiple programs areactive, but they do not permit viewing or control of the underlyingprogram or data represented thereby. Instead, icons require userintervention, typically in the form of a mouse-click on an icon ofinterest, to view or control the program or information. Consequently,the user's viewing options are limited to a choice between onepresenting very limited information about a multitude of programs andinformation and one presenting full information, but of only a singleprogram or data source.

[0015] The fact that the GUI's of the present art are largely restrictedto icons and windows diminishes the capacity to organize, manage, andaccess available information. With the Internet representing an everexpanding view of currently accessible global information, the need forflexible information management tools has become crucial. Similarly,with the current expansion of television programming available, forexample, through cable television and satellite broadcasting, the needto manage this audiovisual content becomes acute. The convergence oftelevision programming and computers increases these management needsall the more.

[0016] Current computer operating system software utilizes bookmarkingschemes for managing Internet locations and complex databasetechnologies for managing specialist information. Neither providesvisual immediacy or ease of layout. Bookmark hierarchies are presentedas cascading textual menus and database technologies arrange informationinto rigidly defined structures. The missing capability is a visualcategorization in which an area of the display unit itself becomes thebookmark and the arrangement on the display becomes the categorization,independent of the type of content.

[0017] While the most common way of accessing information sources is viaa personal computer, present day technology exists to communicate via atelevision or set-top box, handheld computing device, mobile andcellular telephones, all manner of “hybrid” devices such as aninternet-accessible games player, or even in-car navigation and securitysystems, in which case Internet content and other data can be displayedas some portion of the screen. There is a growing convergence oftechnologies: televisions are beginning to find application as viewersof non-television data, (for example through use of “Vertical BlankingInterval” technology in which a signal is inserted into the main videosignal or through set-top boxes providing limited computer andcommunications functionality); computers are already finding applicationfor the display of movies, real-time data streams, and the playing ofaudio data; handheld computing devices and mobile telephones are alsobeing enabled to access the Internet and other information sources; andother devices are becoming able to access a variety of signals, e.g.,digital signals from satellites such as digital radio broadcasts.

[0018] To summarize the current state of the art, display technologiescurrently lack an interface which is capable of organizing any type ofinformation, and presenting such information to the user in a consistentmanner, in such a way that all currently open channels are able toindicate their activity on a continual basis, and which could run on anydevice. Furthermore the current art is deficient in respect oforganizing multiple sources of information in a relevant manner thattakes into account relationships between the various sources.

SUMMARY OF THE INVENTION

[0019] Accordingly, the present invention provides an easy to usegraphical interface that facilitates the organization and management ofmultiple data sources corresponding to a user's needs and interests. Thepresent invention comprises a grid of tiles that resides on the user'scomputer desktop. The grid of tiles provides a uniform, graphicalenvironment in which a user can access, operate, and/or control multipledata sources on electronic devices. Additionally, the invention providesfor automatic control of the refresh rates of multiple data sources,based on the character of the data or on “quality of service tags” thataccompany the data. Such automatic control may operate in conjunctionwith one or more user preferences for the refresh rates of certain data.The graphical environment is uniform with respect to the source ofaccessed information and can manage multiple streams of content,entirely of the user's choice. For example, the invention presents videoclips, e-mail messages, television shows, Internet sites, applicationprograms, data files and folders, live video streams, music, radioshows, and any other form of analog signal, digital data orelectronically stored information, to the user uniformly andsimultaneously, regardless of whether the information is stored locallyor available via modem, T1 line, infrared, or any other form ofcommunication. The user's impression of the interface is alsoindependent of the type of electronic device upon which it isimplemented.

[0020] The present invention comprises a method executed by a computerunder the control of a program stored in computer memory, said methodcomprising the steps of: partitioning a visual display of a computerinto an array of tiles in a non-overlapping configuration; assigning afirst refresh rate to a first tile of said array of tiles and a secondrefresh rate to a second tile of said array of tiles; updatinginformation presented to said first tile in accordance with said firstrefresh rate; and updating information presented to said second tile inaccordance with said second refresh rate. The manner of assigning thefirst and second refresh rates may depend upon the type of informationreceived by each tile, including but not limited to the informationcontent or an identifier contained in, or associated with, theinformation.

[0021] The present invention additionally includes an electronicreadable memory to direct an electronic device to function in aspecified manner, comprising: a first set of instructions to controlsimultaneous communication with a plurality of datastreams: a second setof instructions to partition a display into an array of tiles; a thirdset of instructions to associate a first datastream of said plurality ofdatastreams with a first tile of said array of tiles and a seconddatastream of said plurality of datastreams with a second tile of saidarray of tiles; a fourth set of instructions to retrieve data from saidfirst datastream in accordance with a first retrieval rate and retrievedata from said second datastream in accordance with a second retrievalrate; and a fifth set of instructions to present data to said first tilein accordance with said first retrieval rate and present data to saidsecond tile in accordance with said second retrieval rate. The first andsecond retrieval rates may be automatically assigned by the fourth andfifth set of instructions respectively, based upon the type ofdatastream or an identifier that accompanies each datastream, with orwithout the additional use of one or more user-specified retrievalrates.

[0022] The present invention additionally includes a system forfacilitating the organization and management of multiple data sources,comprising: a device that includes a processor configured to executeinstructions, a memory connected to the processor to store at least oneprogram that includes a graphical user interface, and an input devicethat includes a display, wherein the processor executes instructions to:control simultaneous communication with a plurality of informationsources; partition the display into an array of tiles; associate a firstinformation source of the plurality of information sources with a firsttile of the array of tiles and a second information source of theplurality of information sources with a second tile of the array oftiles, such that information from the first information source isdisplayed on the first tile and information from the second informationsource is displayed on the second tile, wherein information from atleast one of the first source and the second source contains anidentifier; retrieve information from the first information source inaccordance with a first retrieval rate and retrieve information from thesecond information source in accordance with a second retrieval ratewherein the first and second retrieval rates are allocated based uponthe identifier; and present information to the first tile in accordancewith the first retrieval rate and present information to the second tilein accordance with the second retrieval rate.

[0023] The application program of the present invention runs on manydifferent devices, including, but not limited to set-top box, personalcomputer and hand-held device. The grid and tiles retain the samecharacteristics, regardless of operating device. For example, the tilesremain individually configurable and can offer near real-time views oftheir data content. The application therefore permits the user'sinteraction with a range of electronic devices to be unified.

[0024] Additionally it is to be understood that the application programor programs of the present invention may be distributed between a clientdevice and a server in such a way that certain data storage andintensive functions are carried out on a server.

BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Additional objects and features of the invention will be morereadily apparent from the following detailed description and appendedclaims when taken in conjunction with the drawings, in which:

[0026]FIG. 1 shows a representative embodiment of the user interface ofthe present invention comprising a grid of tiles as might be depicted ona display screen.

[0027]FIG. 2 depicts a system that, in accordance with the presentinvention, accepts data in at least one form through at least one portand which additionally displays data to a user.

[0028]FIG. 3 shows stylized examples of an icon and an applicationwindow as are commonly found in computer display systems of thebackground art.

[0029]FIG. 4 shows several tiles as might be found in a typicalembodiment of the present invention.

[0030]FIG. 5 shows an exemplary data structure of the tile object withinthe graphical user interface of the current invention.

[0031]FIG. 6 shows one embodiment of a tile in markup language.

[0032]FIG. 7 shows the hierarchy of software objects underlying thecurrent invention, comprising a grid object, tile objects and files orapplication software.

[0033]FIG. 8 shows an exemplary layout of the display produced by thecurrent invention.

[0034]FIG. 9 shows an alternative exemplary layout of the displayproduced by the current invention.

[0035]FIG. 10 shows an exemplary layout of the display of the currentinvention wherein a tile contains another instantiation of a grid.

[0036]FIG. 11 shows an exemplary layout of the display of the currentinvention including specific examples of tile contents.

[0037]FIG. 12 shows the data structure of the grid object which formspart of the graphical user interface of the current invention.

[0038]FIG. 13 shows one embodiment of a grid in markup language.

[0039]FIG. 14 shows a sequence of windows that demonstrate how a gridmight be set up for initial use by a “wizard” tool in one embodiment ofthe present invention.

[0040]FIG. 15 shows an example of the architecture of the computerprogram in a preferred embodiment of the present invention.

[0041]FIG. 16 shows the architecture of the application program and itscomponents in a preferred embodiment of the present invention.

[0042]FIG. 17 shows the architecture of components of the computerprogram in a preferred embodiment of the present invention.

[0043]FIG. 18 shows an outline of the widget-set used in a preferredembodiment of the present invention.

[0044]FIG. 19 shows an outline of the metabase according to a preferredembodiment of the present invention.

[0045]FIG. 20 shows an outline of the XP Core and its interaction withthe operating system library in a preferred embodiment of the presentinvention.

[0046]FIG. 21 shows an overview of the event system utilized in apreferred embodiment of the present invention.

[0047]FIG. 22 shows an overview of the connection layers that areresponsible for controlling the download of multiple web-pages from theworld wide web.

[0048]FIG. 23 shows a number of functions used by the bandwidthcontroller.

[0049]FIG. 24 is a schematic representation of the relationship betweena server and a client device.

[0050]FIG. 25 shows a series of interactions between a client device anda server.

[0051]FIG. 26 shows how a user, a server and third party contentproviders communicate in accordance with an embodiment of the invention.

[0052]FIG. 27 shows an embodiment in which the application programcommunicates with one or more wireless devices.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0053]FIG. 1 shows an illustrative configuration of the graphical userinterface of the present invention. A grid 1 consisting of a 3 by 3matrix of nine tiles demonstrates some of the different contents thattiles can display. Tile 10 points to a database of stock quotes. Tile 20displays the active folders in an electronic mail utility. Tile 30displays a portion of an alphabetical list of quoted companies. Tiles40, 50, 60, 70 and 80 point to websites displaying, respectively, hightechnology news, electronic goods for sale, categories of business news,items available by auction and the Wall Street Journal. Tile 90 pointsto the file-viewer of a windows-based operating system, such asMicrosoft Windows™, and displays the currently accessible disc drives.

[0054] Within the scope of the present invention, an information sourcemay comprise any analog signal, source of digital data or a datastream,including, but not limited to, video, audio, text and graphics. Theinformation may be in any format, including but not limited to ASCII,bitmap, MP3, JPEG, GIF, TIFF, a mark-up language such as HTML, XML,VRML, HDML, formatted text such as rich text format, or binary.Preferably the software of the present invention is able to recognizethe type or format of the information source and assign properties totiles according to the type. In another preferred embodiment, theinformation source contains or is accompanied by an identifier thatindicates a priority to be attached to the information when displayed bythe graphical user interface. Such an identifier may be referred to as a“quality of service”, (QoS) tag.

[0055]FIG. 2 is a general representation of a data display system 100within which the present invention may be implemented. System 100comprises a processor such as central processing unit 104, an inputdevice 106, data connection ports 108-1 through 108-N, a display 110 anda main memory 112, all connected via bus 116. Residing in the memory 112is an operating system 120, a file system 122, a cache 124 for temporarystorage of information, application programs 128-1 through 128-N and agraphical user interface (GUI) program 126 that is responsible forpresenting information on to display 110. Information may enter thesystem through any one of the ports 108-1 through 108-N which maythemselves be connected to a tuner 114, or via a network interface 134to a communications network. If a tuner 114 is employed, it may channelinput from a wireless signal 130 or a cable network 132.

[0056] In one embodiment of the present invention, system 100 is apersonal computer such as a desktop workstation or a portable notebookor laptop computer. In that case the input device 106 may be a keyboard,mouse, trackpad, trackball, or any combination thereof and the display110 may be a conventional cathode ray tube (CRT) or flat-screen displaysuch as an active matrix or an OLED. The network interface 134 may thenbe a connection to the internet or to a local area network via a cableand modem or a digital subscriber line or ISDN.

[0057] In another embodiment of the present invention, system 100 is amobile or cellular phone or personal digital assistant and the inputdevice 106 may consist of several buttons on a keypad, a touch-sensitivescreen with a touching device such as a stylus, or a microphone andvoice-recognition software. In this embodiment, the display 110 ispreferably an LCD screen or an electroluminescent display and ports 108receive data from radio signals or a portable or wireless modem.Consistent with this embodiment, it is also possible that system 100 is,or comprises part of, a hybrid device such as an internet-readyentertainment device that also has a games playing function or theability to display an electronic book, or has a digital audio functionas does an MP3 player. It is also possible that system 100 is a portablenavigation system, such as a GPS device, especially one that has anadditional functionality such as cellular telephone communicationcapability or the ability to pick up broadcast radio or TV signals.

[0058] In yet another embodiment of the present invention, system 100comprises a set-top box wherein display 110 is a TV screen or monitor,and tuner 114 accepts input in the form of a wireless signal 130 frombroadcast transmissions or cable signals from the cable network 132.

[0059] It is consistent with the present invention that, as discussedhereinbelow, system 100 comprises a client device that communicates witha server.

[0060] It is also envisaged that the present invention can comprise adevice that accepts digital signals from a satellite, such as an “XM”radio transmission. It is further contemplated that the presentinvention can interface to an in-car navigation and security system thatutilizes GPS technology or satellite communication means, such as the“OnStar” system. Accordingly, the present invention may also findapplication in conjunction with an in-car entertainment and displaysystem.

[0061] Input device 106 may be a hand-held remote control apparatus orbuttons located on the set-top box or touch-sensitive areas of display110. Accordingly, the display 110 and the input device 106 need not bepart of the same physical device as the device that contains processor104.

[0062] According to the present invention, the processor executesinstructions to: control simultaneous communication with a plurality ofinformation sources; partition the display into an array of tiles;associate a first information source of the plurality of informationsources with a first tile of the array of tiles and a second informationsource of the plurality of information sources with a second tile of thearray of tiles, such that information from the first information sourceis displayed on the first tile and information from the secondinformation source is displayed on the second tile, wherein informationfrom at least one of the first source and the second source contains anidentifier; retrieve information from the first information source inaccordance with a first retrieval rate and retrieve information from thesecond information source in accordance with a second retrieval ratewherein the first and second retrieval rates are allocated based uponthe identifier; and present information to the first tile in accordancewith the first retrieval rate and present information to the second tilein accordance with the second retrieval rate.

[0063] As is known to those skilled in the art, a graphical userinterface is a computer program that resides in memory 112 of some dataprocessing system and that provides means for presenting information,input to and output from application programs 128 or content ofdatastreams from ports 108 on an associated display. In the backgroundart each datastream is associated with a window. The graphical userinterface allows a user to control the arrangement and display format ofthe data content in each window. Usually a graphical user interfacepermits a user to specify and alter operating parameters of applicationprograms running on the system, though at any given time a particularapplication program will take priority, meaning that a particular windowwill be displaying continually updating content. Typical operatingparameters that may be changed depend upon the application program butinclude the number of buttons on a tool bar and number of visibletoolbars, the size of the text displayed and the color of thebackground.

[0064] By contrast, the graphical user interface of the currentinvention not only permits a user to control the layout of the datacontent but to prioritize each application program running on the systemand each datastream of interest. A novel feature of the presentinvention is that the data content of any number of the programs canvary in real time and the rate at which the display of each is updatedcan be controlled by the user. Furthermore, in another embodiment, inthe absence of initial preferences specified by the user, the presentinvention is able to assign a rate at which the display of a tile isrefreshed according to the type of data or according, for example, to anidentifier that is presented with each datastream or source ofinformation.

[0065] When the present invention assigns a refresh rate to a tileaccording to the type of data, it is envisaged that the type of the datais automatically recognized or ascertained. For example, according tothis embodiment the present invention distinguishes between types ofdata such as video, audio and web-based content, based on the format ofthe datastream in question and preferably assigns a higher priority totypes of data that vary in real time, such as video or audio content.

[0066] When the present invention assigns a refresh rate to a tileaccording to an identifier presented with each source of information,the nature of the identifier may vary according to the type of data orthe protocol that is employed for its transmission. In preferredembodiments, the identifier is a tag that is associated with a “Qualityof Service” (QoS) implementation. As is known to one of skill in theart, QoS refers to the ability of a data communications system to defineand differentiate levels of performance. For example, in internetprotocol (IP), QoS tags are presented as specific pre-set bits in packetheaders. When the packets are received, a priority can be assignedaccording to the bits that have been set. For a description of QoS inthe IP context, see, for example,www.ittc.ukans.edu/˜rsarav/ipqos/ip_qos.htm. A finer granularity of QoSis possible using Resource Reservation Protocol (“RSVP”). For adescription of QoS in wide area networks (WAN), see for example,www.cisco.com/warp/public/cc/so/neso/vvda/avvid/eeqos_ds.pdf. Innetworks that communicate via Asynchronous Transfer Mode (ATM), eachdistinct datastream has an identifier, often called a “virtual channelidentifier,” that accompanies each of its packets (or cells). In apreferred embodiment, the identifier is contained within a headerattached to each packet or cell. QoS is achieved by discriminatingbetween datastreams according to their different identifiers. QoS may becontrolled by adjusting parameters that include a packet loss ratio, apacket cell delay variation, a maximum packet loss transfer delay and amean packet transfer delay. A user's desire in respect of the priorityto be accorded to a particular datastream is communicated amongst thevarious nodes in the network. A more detailed description can be foundin, for example, High Performance Communication Networks, Walrand, J.and Varaiya, P., Morgan Kaufmann Publishers, Inc., (1996).

[0067] Accordingly, an identifier suitable for use by the presentinvention for assigning a refresh rate to a tile includes a bit or bitsin a packet header, or a tag that accompanies or is in the header of acell or packet.

[0068] Consistent with altering refresh rates according to a QoS conceptis that the level of packet loss can be adjusted according to thepriority level associated with different types of data. Furthermore, afully-implemented QoS capability permits prioritization based oncriteria other than the type of the data. For example, traffic can beprioritized according to the IP address of a source or destinationdevice. Traffic can also be differentiated based on whether or not theyare real-time, such as voice or video.

[0069] Tile Objects

[0070] In the ensuing discussion, tile objects are introduced anddescribed and contrasted with existing elements of graphical userinterfaces. A tile presents content from any information source.

[0071] Conventional graphical user interfaces of the background artprovide two distinct representations of programs, files and datastreams,as shown in FIG. 3. One representation is an icon 320, the other is awindow 340. An icon typically occupies only a relatively smallproportion of the available display area and comprises an easilyrecognizable depiction of the program or file, either through its logo322 or some characteristic picture with the name 324 of the programvisible. An icon can be selected by, for example, a touch screenpointer, a cursor controlled by a mouse with a button or by a keyboardstroke or any combination of the foregoing. In response to a furtherselection operation on an icon, for example a double-click of a mousebutton, the graphical user interface will provide a window that can beused to communicate further information to the program or review theassociated datastream.

[0072] A window 340 may occupy a substantial percentage of availablescreen space, usually 90-100%. The window 340 usually comprises a titlebar 348 and a display area 354. The window 340 can commonly be resizedby the user for example by using buttons 352 or a draggable area 356 andhas a format which contains many active areas around its borders.Examples of active areas include a menu bar 342, a vertical scroll bar344, a horizontal scroll bar 350 and one or more tool bars 346. Eachactive area may be used to control aspects of the window's appearance orto set parameters specific to the operation of the program associatedwith it such as text formatting options in a word-processing package orredirection of a web-browser to its stored home location.

[0073] In the present invention, a third graphical representation ofprograms and files, herein called a tile, is introduced. Tiles permit“dynamic bookmarking” of information in that each tile is a viewer of asingle information source—including streaming data sources—and can becustomized with the user's choice of content as well as initialized withsets of options that are pre-determined by the software of the presentinvention according to the type of data that the tile displays.

[0074] A tile is different from an icon because it provides a real-timeor near real-time view of the underlying information in that it containscontinually refreshed content. A tile is different from a window becausea tile will typically be smaller in size than a window, allowing theuser to view multiple tiles simultaneously if desired.

[0075] A tile provides an at-a-glance view of the current status of theprogram or file associated with it but does not necessarily have thelarge number of active areas associated with windows such as title bar,menu bar, toolbars, and scroll bars. Therefore tiles lead to a reductionin clutter on the display screen because many tiles may be displayedsimultaneously without overlapping with one another in the way thatwindows must necessarily do. Tiles are superior to icons because theygive an immediate indication of the current state of the file or programand have utility functions associated with them, as describedhereinbelow. Another advantage of using tiles is a uniformity ofappearance between tiles which correspond to different programs anddatastreams. The display content of a tile will differ from applicationto application though its size and format need not.

[0076] A tile is associated with a program, file or datastream, similarto the way in which an icon and a window are. A tile may present data inany of a number of ways. For example, in a preferred embodiment, thetiles may present a miniaturized, “thumbnail” view of the underlyinginformation; a “porthole” view of a portion of the underlyinginformation as viewed at full size; a symbol indicating whether theinformation has been updated since it was last viewed; or a custominterface designed to allow rapid access to the underlying information.The way in which a tile displays content may be independently configuredfor each tile and may be set up in a default manner that depends uponthe type of information being displayed.

[0077]FIG. 4 illustrates representative tiles. Tile 402 displays apicture or graphic such as may be stored in a bit-map, JPEG, TIFF or GIFfile, or on a world-wide web page. The content of tile 402 is typicallya miniaturized representation of a graphic or still-frame from adatastream. Tile 404 displays a portion of a text document or text of aworld-wide web page. In this sense the tile functions as a transparentpanel placed on top of a document, thus permitting a portion of thedocument to be displayed. Tile 406 displays a further array of tilesthat may be displayed in full by expanding tile 406 to occupy the fullarea of the display. Tile 408 has been configured to link to anelectronic mail program. An alarm setting associated with tile 408 hasbeen configured so that the tile displays an envelope and the message“New Mail!” when an unread message has been received by the mailprogram. Tile 410 displays a name, “FM 101”, denoting the title of abroadcast signal, in this case audio, that is associated with the tile.Tile 412 displays a “thumbnail” of the document viewed in a window suchas 340. A “thumbnail”, as used herein, is a miniaturized representationof an image that retains sufficient characteristics to permit easyrecognition of the image. For example, if the document displayed in awindow were a document with multiple pages, tile 412 may display athumbnail of the first page of the document.

[0078] Tiles are selectable and live. When a tile is selected, whetherby mouse click or otherwise, the tile instantly provides the user withaccess to the underlying information, whether that data be ahierarchical menuing system leading the user to a different level ortiles, a word processing file stored on a local area network, aspreadsheet stored locally on a user's computer, HTML file on theInternet, or a television signal. The tiles are live in that eachcontains real-time or near real-time information.

[0079] In a preferred embodiment a selection operation is associatedwith a tile. For example, clicking on a tile will cause it toimmediately refresh its contents. Selecting tile 402 causes the mostrecent frame of the video stream to be displayed by the tile, or, if thepicture were obtained from a static graphic file, causes the most recentversion of the file to be displayed by the tile. Selecting tile 404 ortile 412 causes the most recent version of the document to be displayed.

[0080] In a preferred embodiment, a second selection operation isassociated with a tile. For example, double-clicking a tile causes thattile to occupy a substantial fraction of the whole display area. In thisembodiment, double clicking tile 402 or 412 causes the image to beexpanded to fit the whole display area. In an alternate embodiment,selecting a tile causes it to occupy an area in the middle of thedisplay that is larger in area than a single tile but does not occupythe full display. If tile 404 were double-clicked, as much of thedocument as could be displayed in the enlarged tile in regular font sizebecomes visible. Double-clicking tile 408 causes the mailbox window ofthe e-mail utility to be sufficiently enlarged to allow new messages tobe selected and read. Double-clicking tile 410 causes the audio streamto become audible over the appropriate channel of the system and neednot necessarily cause the tile to occupy a substantial fraction of thedisplay area.

[0081] A representative tile data structure 500 is shown in FIG. 5. Itis important to understand that the tile itself is an image that at anygiven instant is resident on the file system. This image is separate anddistinct from the application program or file associated with the tile.The tile data structure 500 comprises two addresses: a tile address 502that defines the location on the file system where the tile image isstored; and a target address 504 that is the location at which the fileor application program associated with the tile can be found.Additionally, the tile data structure contains a name 506 that may bedisplayed on the tile in certain circumstances. Tiles of the presentinvention may be assigned at least seven functions, including but notlimited to: an initialization function 508 that is responsible forestablishing a connection with the target address 504; a refreshfunction 510 that handles updates to the tile image stored at the tileaddress 502; a screen-size function 512 that stores the dimensions ofthe display area filled by the tile upon receipt of a request; an alarmfunction 514 that permits the tile to display an alarm or warning whenthe application program associated with the tile encounters a designatedevent; an on mouseover function 518 and an on mouseout function 520which control the behavior of the tile when a selection tool such as amouse-controlled cursor is placed respectively on and off the tile; anda toolbar function 522 which may permit an array of special buttons toappear on or adjacent to the tile for the purposes of adjustingproperties of the tile.

[0082] Refresh function 510 is able to indicate a retrieval rate atwhich information is retrieved from a source of information so that theinformation is displayed and refreshed in the tile. Preferably refreshfunction 510 is able to interpret an identifier that accompanies theinformation displayed by the tile. In one embodiment, a tile isconfigured so that a right-click of a 2-or 3-button mouse while thecursor is over the tile would activate the tool-bar function. In apreferred embodiment, right-clicking on a tile can reveal a menu ofoptions that enables the user to ascertain properties of the tile, suchas the bandwidth it is consuming.

[0083] In one embodiment of the present invention, the tile is itself adocument created in a markup language such as HTML or XML as shown inFIG. 6 and is suitable for display in a web-browser. In this embodiment,a tile occupies a position in a table defined with elements of mark-uplanguage which will be familiar to one skilled in the art. Tile-specificattributes are introduced which control the way in which a web-browserdisplays the tile. In the example in FIG. 6, the tile has a clickablemap, i.e., separate areas of the tile produce separate results whenclicked. Also, this tile has a toolbar, which in one embodiment mayappear if the mouse is right-clicked when the cursor is over the tile.

[0084] In another embodiment of the present invention, tiles communicatewith one another and have conditional content. That is, the content ofone tile depends upon the content of another. Such conditional contentenables more than one tile to be refreshed simultaneously, or nearlysimultaneously, by requesting a given tile to be refreshed.Alternatively, a tile may be set up to specify one or more other tileswhose contents are linked to it. For example, a first tile may show aweb-based listing for the Nasdaq Stock index, and one or moreconditionally dependent tiles may link to web pages that show theweekly, and yearly fluctuations of the index and the performance ofpreviously chosen stocks.

[0085] In a preferred embodiment, tiles are categorized into a number oflevels, arranged hierarchically so that tiles in each level havepriority over those in other levels. Such a priority indicates thattiles with highest priority will refresh most frequently. Accordingly,each tile has a tile level 524 in this embodiment. In one aspect of thispreferred embodiment, priorities are assigned according to the type ofdata that is displayed by the tile. According to such an aspect, thehighest priority is assigned to a video or web-based conference, forexample; the next priority is assigned to a videostream such as a movieor broadcast TV signal; a third priority is attached to a media playersuch as RealPlayer; and lower priority tiles handle local applicationsand files. Tiles within a given level can be set up to compete equallyfor available system resources and bandwidth.

[0086] Grid Object

[0087] The arrangement, layout and independent functioning of the tileson the display and exemplary software for their control is now describedwith respect to FIGS. 7-13.

[0088] The overall hierarchy of a graphical user interface embodied bythe present invention is summarized in FIG. 7. The Grid 700 is the toplevel functionality to which application programs are subordinate. Thegrid can replace the functionality of a user's computer desktop andoffers similar and additional features. Grid 700 comprises a matrix oftiles of which tiles 702, 704, 706, 708 and 710 are representative. Theuser can control the grid in such a way that the tiles 702, 704, 706,708, and 710 present simultaneous information content from a pluralityof sources. The grid controls the layout and priorities of the tiles.Each tile is associated with a data stream or application programthereby permitting a number of different images to be displayedsimultaneously. In particular, different tiles can be associated withdifferent contents. For example, tile 702 is connected to a web-pageviewer 712 such as a browser. Tile 704 is connected to a source ofstreaming video 714, such as Real Player. Tile 706 is connected to anaudio player 716 such as a CD-player program or a source of streamingaudio such as Real Audio. Tile 708 is connected to a file viewer 718such as a text-editor or a word-processing package. Tile 710 isassociated with another grid object 720, thereby permitting a “layering”of information hierarchies. In a preferred embodiment, a grid embodies asimilar underlying data structure to a tile.

[0089] Each tile is separately associated with a source of information,for example, an application program, datastream or file, any one ofwhich may be another grid object. Such a hierarchical structure permitsa user to organize programs and information through the graphical userinterface. For example, separate categories of information can bedisplayed on separate grids allowing each grid to be associated with atheme. In one embodiment, a grid can be configured to be populated witha fixed number of popular web pages based on, say, most recently visitedURL's, or most frequently visited URL's over a period of time such as aweek or a month.

[0090] In a preferred embodiment, a source of information has anidentifier, such as a quality of service tag, associated with it. Thegrid assigns a priority to a tile based upon the identifier, or basedupon the type of data that the source of information comprises. Wheretiles are ranked into levels, the grid assigns an information source toa tile in a level that is appropriate for the type of data or theidentifier associated with the information source. In this way, a tilecan be automatically given a priority that is appropriate for the typeof information that it is to display.

[0091] By using the native attributes of a tile, a user may specify apresentation of the grid, consisting of its dimensions, (i.e., thenumber of tiles to display and their arrangement), and the programs orfiles to be associated with each tile. A single grid, composed ofmultiple tiles, may therefore present a number of information sourcessimultaneously.

[0092] Together, the grid and tiles comprise the application throughwhich a user can view simultaneously information from a multitude ofavailable sources including multiple sites on the Internet or some otherdistributed computer network, receive signals from multiple broadcastchannels, and open and view multiple files. In its initial embodiment,the application may be run through conventional computer operatingsystems, whereupon it overlays the user's desktop and acts as if it werea “borderless browser”. Therefore the application resides over existingapplications without replacing any of them; rather it enables them to becalled from the grid itself. The application, therefore, becomes agraphical file manager in which the content of continuously changingfiles, e.g., datastreams, is being displayed in real-time or nearreal-time, depending on the assigned priority. Effectively, theapplication can be used instead of the user's computer “desktop” becauseit has a more visually intuitive dynamic menuing system than atraditional desktop.

[0093] In a preferred embodiment, a grid of tiles replaces thefunctionality of the computer “desktop” utilized by many modern computeroperating systems. Whereas a desktop is typically populated by staticicons and tool bars, a grid of tiles instead presents to the user anarray of snapshots of current programs and files. The essence of thegrid is that its content is dynamic and informative. As previouslydiscussed, icons are inherently limited in the information that they canpresent and windows clutter the entire desktop without permitting morethan one to be readily displayed simultaneously. By contrast, each tileon the grid can show the current status of the data or datastreamassociated with it. The fact that a tile may refer to a separate gridpermits nesting of grids and consequently a hierarchy of organizedinformation sources.

[0094] The grid also understands the interests of the user and acts as arepository for passwords and identifiers to subscription services. Inthis way, it is not necessary for a user to remember and enter a username and password to access data requiring such a log-in. In this samevein, Internet sites of interest can be “bookmarked” and stored by thegrid, each such site possessing its own tile. The grid is also nestablein that a tile in one grid may point to a separate grid and tiles in theseparate grid may point back to the first grid or to yet more grids. Ifdesired, a user can impose a “theme” on a grid and thereby categorize,group, and/or otherwise manage his data sources. In this manner, theuser can group tiles relating to particular subject matters, Internetsites, documents, or otherwise in a grid according to some speciality. Atile in such a grid could link to another grid to provide a connectionbetween related categories. Equally, a tile on one grid can point to thesame information source as a tile on another grid.

[0095] In a preferred embodiment, the grid permits a regular layout oftiles on the display screen such that the tiles are uniform in size andshape, as depicted in FIGS. 8-11. Each tile is indexed by its positionon the grid. For example 802-2-1 is the first tile in the second row.Tiles in the first row of the grid are 802-1-1, 802-1-2, 802-1-3 and soon, to 802-1-N. Tiles on the second row are 802-2-1, 802-2-2, 802-2-3and so on, to 802-2-N. And, tile the bottom row are 802-M-1, 802-M-2,802-M-3 and so on to 802-M-N. There are no gaps between the tiles, thetiles are not permitted to overlap and the whole grid is covered bytiles. FIG. 8 shows one embodiment in which all tiles are the same sizeand are presented in an array comprising M rows and N columns. There isno particular requirement that the arrangement consists of more than onerow and more than one column. On the display of a mobile telephone orsmart phone, for example, a single row of tiles may be apposite. In apreferred embodiment, a display of a smart, cellular or mobile telephonepresents a single tile at any one time but it is possible to scroll fromtile to tile across the grid, by using, for example, buttons on thephone, or other methods of control.

[0096]FIG. 9 shows an arrangement in which there is a unit tile size,that of tile 802-1-1, but tile 802-1-2 and tile 802-M-1 have each beenconfigured to occupy regions of the grid equal to exact multiples of theunit tile size. Such an arrangement may be useful and important if oneor more datastreams is of particular interest but the others are also tobe monitored at the same time.

[0097]FIG. 10 shows an arrangement of tiles in which Tile 802-M-1 isassociated with a further grid 1000. The lower half of the figure showsan enlarged perspective of that tile showing a grid with Y columns and Xrows.

[0098]FIG. 11 shows an arrangement of tiles in which are depicteddifferent application programs associated with three of the tiles. Tile802-2-2 links to a file viewer displaying a specific file; tile 802-M-1presents streaming video; Tile 802-M-N depicts a page of information onthe world wide web.

[0099]FIG. 12 shows a schematic data structure of the attributes of oneembodiment of grid object 700. The architecture shown in FIG. 12 appliesto any grid, including a top-level grid 700 shown in FIG. 7 as well asto a grid that is contained within a tile.

[0100] Significantly, the grid manages the flow of information to thetiles. For example, the grid can communicate with the display device inorder to determine its current configuration and allocation ofresources. In one embodiment, using a so-called “carousel” approach, thegrid continually cycles around the currently displayed tiles, one byone, refreshing the content of a tile each time it is accessed. When agiven tile is refreshed, the refresh operation is completed beforerefreshing the next tile in sequence. In this way, the cycling rate maybe set so that the current content of all tiles are reasonably up todate. The cycling can be interrupted by a user selecting a given tile,so that that tile alone becomes continuously updated. In this way, theuser does not need to worry about manually refreshing multiple tiles.

[0101] In a preferred embodiment, according to priorities that may beapplied to individual tiles on a tile by tile basis if desired, the gridmanages the refresh rate of each tile in the grid. For example, forlocally stored word processing or spread sheet files, the user mightconfigure the tiles to refresh only when the underlying data is writtento the local hard drive. Similarly, a user might configure tilescontaining infrequently updated Hyper-Text Markup Language (“HTML”) datafrom the Internet to refresh at a certain time each day. At the otherextreme, a user might configure an active tile to display a televisionchannel at a refresh rate of 29 frames per second, while at the sametime configuring inactive tiles to display different channels at arefresh rate of once every five seconds. In this way, a user couldmonitor many channels until program content of interest appeared in oneof the tiles without the burden of actively refreshing each tile.

[0102] In another embodiment, the grid itself assigns priorities to thetiles based on identifiers such as quality of service tags that areassociated with the information source itself. In still anotherembodiment, the grid automatically assigns priorities to the tiles basedupon its recognition of the type of data that is presented to it. Inthis way, the grid can associate information sources with tiles thatoccupy different levels of priorities in a hierarchy of levels, ifpresent. It is to be understood that an identifier such as a QoS tag canbe used to assign to a tile a priority that is higher than that ofanother tile that would display the same type of data, and would havethe same priority in the absence of such an identifier. Thus, forexample, QoS tags can be used not only to assign two videostreams ahigher priority than a web-browser tile, but can be used to prioritizeone videostream over another. It is also to be understood that not allinformation sources need be accompanied by identifiers for the grid tobe able to assign priorities itself. In the absence of an identifier fora particular information source, the grid can assign a priority based onthe type of data, a user preference or some arbitrarily low prioritybased on other considerations such as bandwidth and availability ofother system resources.

[0103] The grid itself has an address 1204 that specifies its locationwithin the file system of the device in which the application programprogram runs. The grid has associated with it several utility programs:a configuration wizard 1206 that may be called by the user when settingup a new grid; a tile creation function 1208 utilized by theconfiguration wizard when initializing new tiles; a tile annihilationfunction 1210 utilized in case of error or when resizing the grid.

[0104] The grid object stores the number of rows 1212 and the number ofcolumns 1214 of tiles that are present. The grid also stores a tile list1216 containing attributes of each respective tile. In particular, theaddress of each tile, its priority and its refresh rate are stored bythe grid program. The grid also stores other attributes of tiles such astheir respective positions on the grid as given by their column and rownumber. The priority of a tile may be used to determine its refresh ratein one embodiment of the present invention. A tile can have a passwordfeature built into it if it is desired to restrict access to the tile'scontent. The grid itself can have a toolbar by which its attributes maybe accessed and modified.

[0105] In a preferred embodiment, a grid is a special form of a tile. Itis a tile that can create and manage an array of other tiles.Accordingly, its data structure also comprises those elements of a tiledata structure shown in FIG. 5 in addition to those shown in FIG. 12. Ifthe grid has a parent grid, the address of the parent grid 1202 isstored. For example, grid 1000, associated with tile 802-M-1 of FIG. 10,has grid 700 as its parent.

[0106] The grid can be configured to contain any number of tiles, fromone to as many as can reasonably fit on the user's display.

[0107] In one embodiment of the present invention, the grid is adocument created in a markup language, such as HTML, SGML or XML, and istherefore suitable for display via a web-browser. In this embodiment,the addresses of the parent grid, the grid itself and each of the tilesare expressed as Uniform Resource Locators (URL's). The variousfunctions controlled by the grid are accessible through function callsdevised according to methods familiar to one skilled in the art. Forexample, “dynamic HTML”, java applets or simple CGI-scripts can providethe technological basis for enabling various grid utilities. FIG. 13 isillustrative code for establishing a grid.

[0108] The application program may be downloaded from a pre-determinedweb-site and preferably operates in a client-server mode. Users maydownload preconfigured grids from the predetermined server. A gridconfiguration “wizard” program which guides a user through a step bystep set up of a custom-grid may also be downloaded. Other web hosts areable to deliver content to end-users via the predetermined server. Somebasic functions of the grid can be carried out on the predeterminedserver and provided to the user. According to the capabilities of thedevice on which the grid is displayed, the server may take on moredemanding functions.

[0109] Grid Configuration Wizard

[0110] In one embodiment of the present invention, the set up of aparticular grid is achieved through a grid configuration program(“wizard”) that is downloaded to the display device from a remote site.The grid configuration program permits a user to define and install oneor more grids on the client system. When a tile is partitioned into afurther array of tiles, the grid configuration program can also be used.One embodiment of the user interface of the grid configuration wizard isshown in FIG. 14.

[0111] A first screen displayed by the grid configuration wizardcomprises an application program logo 1404, button 1406 to guide theuser to the next screen and a number of choices, such as 1408. The useris offered the choice of preconfigured, or “standard” gridconfigurations, selected from a list. Examples of such grids includegrids themed by content such as sport-related grids or by type of datasuch as grids whose content is video-based.

[0112] Additionally, the user is permitted to configure “customized”grids in which each tile can be taken from a list of predefined samplesor can be initialized according to the user's wishes. For example, auser can assign a priority to a tile, or can select a tile from ahierarchical scheme of pre-defined priorities, or can specify that atile set its own default priority by recognizing the type of data thatit receives or an identifier such as a QoS tag associated with thatdata. In a second screen 1412 displayed by the grid configurationwizard, a tiled area 1416 represents the grid that the user is building.Sample tile categories 1418 such as “weather”, “news”, “stocks”, or“sports” are listed. In an alternate embodiment, a grid can be filled bythe “drag and drop” technique in which a selected document 1414 is movedon to the display area of the grid configuration program andautomatically becomes a tile. A button 1410 offers the user the chanceto go back one screen.

[0113] In a third screen 1420 displayed by the grid configurationwizard, the user can name the grid and, optionally, store it for futurereference, for example in an archive of preferred grids. The user canelect to finish grid construction by clicking the “Finish” button 1412,or launch the grid immediately by activating button 1422. When launchingthe grid immediately, the grid is automatically constructed on the flyaccording to the content and tile types specified by the user during theset up procedure.

[0114] Architecture of Application Program Software

[0115] The hierarchy of the software in a preferred embodiment of thepresent invention, the Surfcast System, is illustrated in FIG. 15. It isunderstood that the Surfcast System is merely an exemplary embodimentand that other software products can be developed that fulfill thefunctions of the present invention. It is further understood that theSurfcast System or other equivalent systems, may be distributed betweena client and one or more servers and that all parts of it need notreside on a single device.

[0116] The Surfcast System software comprises a number of modules. InFIG. 15, an arrow connecting two modules means that one module uses aninterface of the other. The arrow comes from the module invoking theinterface towards the module whose interface is being invoked. Aninterface may simply be a function call between the two modules or, forexample, a call to a dynamic linked library (DLL).

[0117] The Surfcast application program 1500 takes its underlying datafrom two sources, a metabase 1506 and a system library 1514 thatpreferably resides on the device on which the application program isrunning. For example, the application program 1500 may be required tocall functions from the system library while it is running. Actual tilesthat a user visualizes can be spawned from the metabase 1506.

[0118] Data structures for user tiles 1502 are obtained from the XP core1504 which itself also utilizes components from the system library 1514.In a preferred embodiment there are different data structures for tilesof different priorities. The XP core is an abstraction layer for theoperating system environment. Tiles that utilize components such asbuttons take these components from the widget set 1508. A widget is abasic user interface element such as a button or a text input box. Themetabase also uses an interface of the widget set 1508, and cantherefore use functions within the widget set. The widget set requestsfunctions from the XP core.

[0119] Objects in the metabase 1506 that retrieve content from remotesources such as world wide web pages utilize a connection manager andbandwidth controller 1512. A URL loader 1510 decides whether contentshould be obtained afresh by contacting the connection manager 1512, orfrom content previously stored in cache. Effectively, the URL loadermanages the connection manager, and calls functions within it.

[0120] Underlying all of the Surfcast application program's operationsare functions from the operating system library 1516 that is suppliedwith the device on which the application program is running.

[0121] Each of the objects shown in FIG. 15 is now described in furtherdetail by reference to FIGS. 16-21.

[0122] The Surfcast application program 1500, in FIG. 16, comprises alauncher 1618 and a framework 1620. The launcher opens the program fromscratch whereas the framework is responsible for managing grids andtiles. The user interacts directly with the framework to set up apreferred arrangement of tiles on the display. In one embodiment, theframework initially contains a prescribed set of tiles. The frameworkcontrols communication between tiles, for example, in the case ofconditional content.

[0123] Tiles 1502, FIG. 16, are the equivalent of an application in aconventional GUI system. They can be built in C++ using the Surfcasttile builder application program interface (API) using XP core classes,or via a utility such as a custom tile editor or via a script file. Somepredefined tiles are included with the basic Surfcast system including aweb browser tile called a surftile 1602, tiles for contacts andcommunications such as a chat tile 1604 and an e-mail tile 1608.Media-player tiles such as a video tile 1606 are also supplied, as aretiles that interface to commonly used desktop programs. A word tile 1610interfaces to a word processor; excel tile 1612 interfaces to aspreadsheet program. A general content viewer that can compose pieces ofcontent clipped from a variety of sources, layout tile 1616, is alsoprovided. Reg tile 1614 is a general purpose tile that permits a user todefine his own tile type.

[0124] In a preferred embodiment, all tiles have a common base class,and each specialized type of tile has its own class that builds upon thebase class. There are many ways in which specific tile classes can bederived from the common base class. In a preferred embodiment, tiles arecategorized into a hierarchy. There may be a tile class specific to eachlevel in the hierarchy. Functions for specific tile classes are readilyapparent to one of skill in the art. Tiles may additionally berepresented by markup language files and viewed within a web-browserenvironment.

[0125] The data classes in XP core 1504, FIG. 16, comprise base classesand utility classes with which tiles and widgets are built. In general,classes within the XP core describe how tiles can communicate with oneanother and with the overall application program framework 1620. Inparticular, XP core classes are generic and portable, thereby permittingcross-platform capability.

[0126] XP core classes include: tilebase 1622 for the generic class thatunderlies all tiles; tile base view 1624; tile controller 1626; canvas1628; and event classes 1630 for event handling. A view is what a tileuses to draw itself. Tile base view 1624 is the base class for all viewsassociated with a visual object. A controller processes events 1630, forexample mouse moves, clicks, keyboard events and external events. Tilecontroller 1626 is the base class for controllers associated with atile. A canvas 1628 is an area of screen on which to render some image,for example a tile.

[0127] The metabase 1506, FIG. 17, is a local store forplatform-specific implementations of the descriptions of tiles, gridsand other objects. Tiles, grids and content are created, saved andrestored via the metabase. The metabase contains tile type and contenttype registries such as tile registry 1732, and a local database ofgrids and content such as content store 1734 and grid store 1736.Unknown tile and content types can be obtained from remote servers. Tiletypes are abstracted so that if a grid contains a particular type oftile, for example an e-mail tile, the metabase provides whatever isappropriate for the device the application is running on. Items in themetabase are “persistent”, that is they are not saved explicitly but arepreserved from session to session.

[0128] The widget set 1508, FIG. 17, comprises a platform-specific setof visual components that tiles can use. Widget set contains thepredefined widgets that are included with the system. It can be extendedwith new widgets. It includes such useful widgets as a button 1738, alabel 1740, an edit widget 1742 that enables a user to enter text intoan editable field and a list widget 1744 that enables a user to selectfrom a set of options. Items within the widget set can be used with tiletypes such as text input tiles, web browser tiles, and a streaming videotile. The term widget can also include more complex objects such as avideo player that can be inserted into a tile as easily as a button.

[0129] The URL loader 1510, FIG. 17, provides a mechanism for retrievingcontent. The URL loader 1510 interacts with connection manager 1512 fortiles which need to make a network connection. Tiles and the metabaseask for content for a given URL and the content manager will attempt toretrieve it. The metabase also contacts the connection manager throughthe URL loader to ascertain whether there is sufficient bandwidth forthe transfer. In particular, the connection manager decides whether theURL loader should furnish tile content from the cache 1746, as would bethe case if the content has been recently displayed and stored locally.Alternatively, if the content is not cached, the URL loader supervisesloading of content from the location specified by the URL.

[0130] The URL loader 1510 also comprises a file manager 1748 fororganizing the cached content. The URL loader additionally comprises aDOM (data object model) renderer 1750 that administers the parsing ofpages in markup languages such as XML and HTML. The URL loader may alsocomprise an implementation of an API for XML rendering such as SAX. Inan alternate embodiment, such an API may reside in the XP core module.

[0131] The system library 1514 comprises commonly used utilities withinthe application program, including, but not limited to, code for stringmanipulations, file handling and server communication. The systemlibrary module comprises generic code and can be compiled for anyoperating system.

[0132] The operating system library 1516, FIG. 17, comprises utilitiesthat differ in their implementation from system to system but are neededfor operation of the application program. For example, utilities thatmay be found in system library 1516 are those that provide support forthreads and synchronization 1752, debugging tools 1754, graphicslibraries 1756, further basic string manipulations 1760 and connections1762. Additionally, not shown in FIG. 17, useful items in the operatingsystem library include utilities that permit definitions of objects,sockets, input devices and hardware devices. Items in the operatingsystem library are accessible through classes with documented publicinterfaces.

[0133] The operation of the widget set 1508 is further described withrespect to FIG. 18. As previously mentioned, the widget set containswidgets for various functions such as buttons, text labels, etc.Illustrated in FIG. 18 is a set of widgets for buttons.

[0134]FIG. 18 shows the class hierarchy for the button widget. In FIG.18, a solid line with an arrow head indicates a relationship ofinherency between two classes. Base classes for widgets are groupedtogether in box 1809. Widget class 1810 is a container for otherclasses. All widgets use the base class widget 1810 stored in the XPcore module and further described later. Accordingly, button 1800 is aspecific class inherited from widget 1810.

[0135] Widget view 1812 is a class, also stored in XP core, that definesthe look and feel of the widget. Button view 1802 inherits from widgetview and controls how a button draws itself.

[0136] In the scheme of FIG. 18, user tile 1816 comprises three objects,tile controller 1626, the base class tile 1622 and tile base view 1624,each of which is found in XP core and is further described later. Tilebase view 1624 is responsible for drawing the tile and can employ one ormore widgets.

[0137] A button is something that a user can click on. A button providesbutton events to a button event consumer. A button event consumer isalso known as a client, i.e., clients that use buttons implement buttonevent consumer 1808 to be notified of button events. For example, abutton event consumer may be a “play” function in a video tile. Thebutton event consumer interacts with a control structure tile controller1626 associated with a user tile 1816 by telling the tile that thebutton has been activated. Button event consumer 1808 is itself a classthat inherits from an event consumer class, described later.

[0138] Button controller 1804 controls how button events 1806 areprocessed, for example mouse and keyboard events. It inherits classstructure from widget controller 1814 stored in XP core. Not all buttonevents 1806 are recognized by the button controller 1804: for example, aparticular key-stroke may have no effect on the state of the button.

[0139] Other widgets follow a similar pattern. They include: textedit,for inputting text; textlabel, for displaying text; textspinner, forselecting from a choice of options; datewidget, for entering a date;list, for selecting from a choice of options; titlebar; and a toolbar.

[0140] Metabase 1506 is further described with respect to FIG. 19.Arrows between the components in FIG. 19 denote interfaces. The metabasecomprises a registry 1900 and store 1902 of tile and grid types, acontent store 1918 and a content and tile link database 1920.

[0141] The tile type registry 1900 contains a list of tile and widgettypes along with information about how to create them and what they arecalled, as well as other information such as pointers to objects fromwhich tiles are created. In a preferred embodiment, the list of tiletypes includes tiles in a hierarchy of categories. The tile and gridstore 1902 contains a library of stored grids and tiles. Tiles can savethemselves or be restored. In a preferred embodiment, grids are justspecial cases of tiles. Tiles from the tile library can be called anddisplayed on the screen by asking the registry to load tiles. The tileand grid store interfaces to an XML library 1906.

[0142] Tiles from the tile and grid store can also be saved outside themetabase in a library of markup language files, e.g., an XML library1906.

[0143] Also in the metabase 1506 is metabase tile 1904 which utilizestile base 1622 from the XP core module. All tiles inherit from thisclass.

[0144] DLLs can contain additional tiles and widgets that can be createdby independent third parties. These can be implemented within themetabase through the tile factory 1916 and tile creator 1914, both ofwhich interface to the tile type registry 1900. The tile factory 1916contains the description and classes necessary for someone to register anew tile type. Tile creator 1914 is the code that does the tile creationat runtime. In general, independent creation of tiles is facilitated bysupplying a tile toolkit to third parties.

[0145] Inquiry utility 1912 is an optional means for an outside user tointerface to the metabase, for example to ascertain the class structuresof stored tile classes.

[0146] Content store 1918 is a cache that contains content of tiles forthe previous session. The content store 1918 follows a similar patternto the tile type registry and the tile store. The content and tile linkdatabase 1920 is a database of information about how tiles and grids arerelated, and how content is related between them. Content tile linkdatabase 1920 copes with descriptions of relationships between tiles andalso themes of content for related tiles. This database can also be usedin the context of “knowledge management”, i.e., those operations thatmonitor a user's activities and attempt to suggest further sources ofcontent based on it. Both content store 1918 and the database 1920interface with tile type registry 1900.

[0147]FIG. 20 shows a further description of classes found in the XPcore and their interaction with the operating system library 1516. Asdiscussed below, some arrows between objects within XP core denoteinheritance of classes.

[0148] Canvas 1628 describes an area of screen that can be drawn to.GfxContext 2002 is a set of graphics primitives, for example forline-drawing, colors and space filling. It is a generic version thatencapsulates and abstracts operating system-specific features inheritedfrom GfxContextFactory 2032 in the operating system library 1516. Win32GfxContext 2034 is an example of a graphics context used with theWindows operating system. Other GfxContent 2036 includes alternativeplatform dependent graphics contexts.

[0149] The foundation classes are tile base 1622, tile base view 1624and tile base controller 2008. Tile base 1622 is the base class for allvisual objects such as tiles, widgets and grids. The tile class 2010,and widget base class 2016, inherit from tile base. Where tiles are in ahierarchy of categories, there may be several tile classes that inheritfrom tile base.

[0150] A widget effectively functions as a special kind of tile that canbe placed inside a tile. Widget base 2016 is not meant to beinstantiated on its own but is a foundation for the widget class 1810,FIG. 18, used by a generic widget.

[0151] Tile base view 1624 is the base class for all views associatedwith a visual object. A view is what a tile uses to draw itself.Inheriting from tile base view are tile view 2012, the base class forall views associated with tiles, and widget view 1812, the base classfor all views associated with widgets. Tile base view and tile base alsointerface with canvas 1628.

[0152] Tile base controller 2008 is the base class for all controllersassociated with a visual object. Inheriting from tile base controllerare tile controller 1626, the base class for controllers associated witha tile, and widget controller 2020, the base class for all controllersassociated with widgets. A controller processes all events. Tile baseinterfaces with both tile base view 1624 and tile base controller 2008.Tile base controller interfaces to event system 2022. Finally, eventsystem 2022 also communicates between the operating system library 1516and the tile base controller 2008.

[0153] Event system 2022 is further described with respect to FIG. 21.An event can be any one of a mouse movement event, another mouse eventsuch as a mouse-click, or a keyboard event such as a keystroke. In FIG.21, event 2112 is the base class. Other classes such as mouse movementevent 2106, mouse event 2108 and keyboard event 2110 derive from thebase class by inheritance. The event consumer 2104 is a classresponsible for directing events to the controller. The event producer2102 interprets system events into Surfcast events for an eventconsumer. The boxes 2122, 2124, 2126, 2128, 2130 and 2132 aremultiplexers handling the case where multiple clients are affected bymultiple types of events. An event is communicated to tile basecontroller 2008.

[0154] Managing Connections to more than one Datastream

[0155] When two or more tiles connect to sources of data available overa network, communication must be established in such a way that the rateat which updated data is transmitted to the grid can be controlled. Inpractice, for an embodiment of the application which resides on a user'scomputer, a flow control protocol such as TCP is required. In this way,each tile can communicate with the remote datastream to which it islinked and a determination can be made of available bandwidth at thetime of data transfer. Alternatively, in a client-server mode, flowcontrol is not necessary because communication with the server suffices,as is described below.

[0156] It is not practical to fire up a separate browser program fromeach tile that wishes to download data from a site on the world wideweb. A web-browser is very greedy on memory and resources and the userwould have little or no control over the respective rates at which datawas downloaded from different sites.

[0157] Instead, in a preferred embodiment of the present invention, ahierarchy of layers manages the simultaneous connection and allocationof resources to different world wide web sites, as shown in FIG. 22. Thelayer structure applies to the way in which any given tile downloadscontent.

[0158] At the highest level there exists a widget, referred to as“SurfWidget” 2200, which is the basic browser control within theapplication program of the present invention. Ideally, this widgetoperates in conjunction with any commonly used world wide web browser.It is typically associated with a tile type such as surftile 1602.

[0159] The surf widget communicates with a surf widget controller 2202in a control layer 2201. Also in the control layer is a web browserapplication program 2204. Examples of such a program include Microsoft'sInternet Explorer and Netscape's Navigator software. The surf widgetcontroller 2202 handles the interaction between the surf widget 2200 andthe web-browser 2204. The surf widget controller also passes on requestsfrom the browser to the URL Manager 2206 in the next layer, the URLlayer 2205. The surf widget controller then pipes back the results to berendered to surf widget. A typical example of this process in operationwould be: a user clicks on a hyper-link in a web-page; the web-browsermakes a request for that page to surf widget controller 2202; therequest gets handed to the URL manager 2206; once the page is loaded,the URL manager 2206 notifies the surf widget controller, which in turnsends the information to the web-browser for rendering.

[0160] The responsibility for obtaining pages of content is that of theURL layer 2205. When a URL is requested, the URL manager 2206 issues arequest for the page and any subsequent media to the connection manager2210. The URL manager keeps track of the requested URL for future use,if it is requested again. The URL manager also queues up URL's that havebeen requested according to their focus, i.e., the tile that a user hascurrently selected and according to the respective priorities of theactive tiles.

[0161] In a preferred embodiment, a pre-fetch utility such as URLpre-fetch manager 2208 can be implemented. It saves the user time ifitems can be pre-fetched instead of waiting for their download. Severalstrategies can be used to obtain pre-fetch items for the user. Using ahistory of a user's previous browsing habits, it is possible to predictwhat the user will probably want next. Another function of a pre-fetchutility is to periodically check the validity of items in the cache tomake sure they are up to date. As would be familiar to one skilled inthe art, some of the new HTTP1.1 methods would prove very useful forthis, namely the conditional gets. Another strategy is to start loadinglinks from the page that a user is browsing, regardless of whether theuser has selected the links. Although such an approach could be verywasteful of resources if there are a lot of links and very few areultimately accessed and also because a lot of links tend to beadvertisers, this approach could be effective in situations where veryhigh capacity bandwidth exists.

[0162] The connection layer 2209 handles each individual request fordownload passed to it through the URL manager, regardless of whether itis an HTML page, a graphic or sound file. The connection manager 2210understands the total bandwidth available for allocation, for example,whether the device is connected to a modem or a T-1 line. It will alsomanage the connection to the requested site and maintain its own cache.Before making a network request for an item, connection manager 2210first checks its cache, the connection manager cache 2212. If the itemis not in the cache, the connection manager then passes the request offto the HTTP protocol socket 2214 in the protocol layer 2215. The way inwhich HTTP protocols and caches work is familiar to one skilled in theart.

[0163] The protocol layer 2215 consists of a suite of different sockettypes, 2214, 2216 and 2218, intended to support different communicationprotocols, such as HTTP, FTP and also a client server protocol specificto the application program via the surfcast protocol socket 2218.Preferably protocols employed are able to recognize QoS tags in theheaders of packets that are passed through the layer.

[0164] The socket layer 2219 comprises at least one socket 2220. Thesocket layer wraps up all the system implementation specifics for agiven platform and allows generic socket types to be built on top. Thesocket keeps track of its bandwidth usage, which can then be queried atthe connection layer. The socket layer then facilitates bandwidthmanagement.

[0165] With all communications going through the same socket layer it ispossible to easily collect data about a socket's bandwidth usage. If, atthe connection layer, it is noticed that the total bandwidth allocationhas been exceeded, it is a simple case of blocking further data transferuntil such time as total bandwidth usage falls back under what has beenallocated.

[0166] As a user switches focus from one tile to another, priorities canbe dynamically re-allocated to ensure the fastest possible loading ofthe selected page. Such an allocation can take into account prioritiesthat have been assigned to the tiles based on their content, as well asbased on identifiers associated with the information such as quality ofservice tags. All other communications can then abide by the same rules,allowing for complete control.

[0167] The sequence of events and functions in a “dynamic bandwidthallocation” feature of the present invention are described as follows.The dynamic bandwidth allocation feature involves a URL loader, theconnection manager 2210 and a bandwidth controller.

[0168] The tiles that need access to the network resource fordownloading content from a URL, pass certain parameters to the URLloader which manages all such requests from the tiles. These parametersinclude the URL itself, the priority of the tile, the minimum bandwidthrequirement if any, and the maximum bandwidth requirement, if any.

[0169] The URL loader detects the need for a connection to a networkresource, as would be notified to it by the connection manager. In thecase of dial-up connections, the connection manager is responsible forallocating the modem resource and making the dial-up. Once a connectionis made and the network resource is available, the URL loader requeststhe bandwidth controller to start delivering the required content,taking into account the additional parameters for each request.

[0170] The bandwidth controller 2300 is an object that comprises anumber of functions, as shown in FIG. 23. AddURL 2302 is a function usedby the URL loader to add an additional connection request to thosealready considered. RemoveURL 2304 is used by URL loader when cancellingor aborting a request. GetURLStatus 2306 is used by URL loader to obtaina status report for a given request. GetStatus 2308 is used to obtain ageneral status report for the overall bandwidth. The bandwidthcontroller 2300 is additionally able to execute a main cycle loop overthe outstanding URL requests for the purposes of managing the bandwidthamong them. The following pseudocode describes steps within the maincycle loop, according to one embodiment of the present invention. while(uncompleted requests outstanding) { calculate bandwidth obtained perrequest; check for need to postpone, stall or cancel lower priorityrequests; check for need to increase higher priority requests; detectcompleted requests and notify requestor; detect undeliverable requestsand reissue or cancel if necessary; carry out other service andstatistics functions, as required; }

[0171] Each of these steps is explained, as follows. One or more of thesteps may be performed in a different order from that presented abovewithout departing from the scope of the present invention.

[0172] The step of calculating the bandwidth obtained per requestutilizes a function that calculates obtained bandwidth for each of themanaged requests, including requests that are stalled (or postponed) dueto priority issues. This calculation takes place frequently because ofthe nature of the network. The bandwidth obtained can vary drasticallyeven during the course of the individual network transaction, andtherefore a priority based dynamic system must continuously accommodatesuch fluctuations. The result of calculating the bandwidth obtained isused both for feedback to users and for making decisions in thesubsequent steps within the main cycle loop.

[0173] The step of checking for the need to postpone, stall or cancellower priority requests is another precautionary mechanism to use forthe purposes of adjusting the currently active connections. Ifoutstanding requests are not achieving the desired minimum bandwidth, oran actual bandwidth in line with the priority of a given request is notachieved, bandwidth must be made available to the higher priorityrequests, by stalling, cancelling or postponing lower priority requests.A throttle feature implemented consistent with the layer nature of thestack can be applied wherein the frequency of issuing requests can bedecreased. A complete cancellation of a request followed by reload fromcache is usually the last resort.

[0174] The step of checking for the need to increase higher priorityrequests has the opposite effect. If applied, higher priority requestsare increased by means of spawning additional simultaneous requests, orby increasing the throttle mentioned above.

[0175] Both the steps of checking the need to postpone and checking theneed to increase a priority can provide feedback to the user in terms oftheir success or failure. Status parameters can additionally becollected and calculated.

[0176] The step of detecting completed requests and notifying therequestor utilizes a function for handling successful completedrequests. Conversely, the step of detecting undeliverable requestsfollowed by reissuing or canceling the request if necessary is themechanism for avoiding undeliverable requests, or retrying temporarilyunavailable requests.

[0177] Other service and statistics functions may also be called, as maybe necessary for supplying information to the layers above theconnection layer.

[0178] Client-Server Interaction

[0179] It is envisaged that the methods of the present invention can becarried out using software that is distributed between a client and atleast one server. At one extreme, as discussed hereinabove, the softwareruns entirely on a server. At another extreme, the client only hosts abrowser that displays the grid and bandwidth allocation decisions areprocessed by a server. It is preferred that the server undertake moreresource and bandwidth intensive tasks than the client.

[0180] In one embodiment of the present invention, FIG. 24, the user atclient device 2400 interacts with server software on a server 2402. Theserver stores locally a profile comprising user-specific content 2406that can feed customized data to the user. For example, the user maystore pre-defined grid configurations on the server. Additionally,passwords for specific web-sites can be stored along with the user'sprofile. A grid generator 2404 on the server creates a grid of tilesaccording to user-specified content. Each tile has been created on theserver by producing an image from the location specified. For example,tile creator 2408-1 produces a tile from content 2410-1. Thus, when auser logs on to the server, for example through a conventionalweb-browser, a grid of tiles is downloaded to the user's system. Wherean identifier is present in content such as 2410-1, tile creator 2408-1can preferably recognize such an identifier and ascribe a priority tothe tile that it creates based on that identifier. In an alternateembodiment, the tile creator automatically assigns a priority to thetile based on the type of the information content.

[0181] The client-server embodiment provides a number of advantages. Forexample, individual users and the devices they use may bedifferentiated. Therefore, the tile-content that the server delivers canbe customized according to the rendering device of a particular user.

[0182] Turning next to FIG. 25, each time a user logs on to the server2402, a “session” is initiated, step 2500. The server verifies theuser's identity, step 2502 and acknowledges the log-in, step 2506. Theclient registers one or more resources, such as connection bandwidth,cost per transmission unit and properties of local playback device, step2504. From this information, the server identifies the client devicetype, for example, set-top box or personal computer, step 2508. At step2510, the server retrieves grid settings specific to the user, ifappropriate. In preferred embodiments the server adjust grid settingsaccording to the type of device, for example the size of its display. Inpreferred embodiments, the server adjusts grid settings according to thetype of device, for example according to the size of its display.Details of a session, as defined by tile content and priorities can beheld over from one session to another, both for the purposes ofpermitting a user to continue with ongoing work and in order to protectagainst the adverse consequences of abnormal disconnections.Additionally, targeted advertising and messaging can be delivered tosubsets of users via the predetermined server.

[0183] Having rendered a grid, step 2512, and delivered it to the clientdevice, the user can request sources of information such as datastreams,step 2518. Any number of datastreams can be requested at once, thecorresponding stream request information which defines parameters foreach stream is communicated to the server which verifies and calculatesparameters for each stream, step 2516. In one embodiment the serverallocates a priority to each stream according to the type of content,e.g., video, audio, static web-page content. In a preferred embodiment,the server uses an identifier associated with a datastream and allocatesa priority based on that identifier. Such an identifier may be a qualityof service tag.

[0184] Upon completion of these steps, the server 2402 knows the clientcharacteristics and is able to distribute bandwidth available to theclient among multiple content servers using for example a bandwidthcontroller 2300. In this example, if the client has an incomingbandwidth of say 56 Kbits/second, and is requesting datastreams fromthree sources with equal priority, then the server will respond for eachrequest that a bandwidth of 56/3=18.7 Kbits/sec is available for eachdatastream.

[0185] The requested datastreams are displayed on the client device,step 2520 and, if the user changes the content of a tile or explicitlyrequests an update or refresh operation on the tile, step 2522, anupdate request is sent to the server, step 2524. Once the new contenthas been received, the grid is rendered anew, step 2526. Preferably therefresh rates of tiles on the client are allocated according topriorities associated with each tile.

[0186] Intensive operations on each displayed tile are also channeledthrough the predetermined server. For example, refresh operations on atile generate a refresh request that is sent to the predeterminedserver. Similarly, requests to thumbnail a given image can be carriedout, by request, on the predetermined server and the resultingcompressed image transmitted to the user.

[0187] In the foregoing embodiment, the server component may residelocally on the client machine, in which case the server is known as the“Resource Manager”, or it may be a remote server.

[0188] In a preferred embodiment of client server operation, shown inFIG. 26, aspects of a user's grid profile are transmitted to thirdparties so that the third parties may then communicate tile basedcontent directly to the user. For example, a user's custom grid maycontain a tile that points to a third party web-site 2604. Content 2606from the 3^(rd) party web-site is typically transferred to the serverfor dissemination to the user. The server 2602 notifies the 3^(rd) partyweb-site that the user requires tiled data by, for example, transmittinguser information 2608. The third party then permits the tile basedcontent of its web-site to be transmitted directly to the user. Thethird party may attach an identifier to its information so that theserver can assign a priority to the data.

[0189] The use of servers also allows for the latest versions of tilesto be downloaded and installed across all devices. Users are then ableto share grids and tiles with other users. The server side technologyprovides users of all client devices, from desktop PC's to mobiletelephones with a consistent experience.

[0190] The majority of the server side code is preferably written inJava, with C++ being used where appropriate. The methods and apparatusof the present invention are not dependent on the particular programminglanguage employed. Inter-server communication also preferably utilizesXML to provide consistency with other aspects of the invention.

[0191] In a preferred embodiment, either Oracle 8i or SQL “Server 2000”are used to provide a relational database (RDB) functionality of theserver. Both of these RDB's now provide direct SQL to XMLtransformations. Databases are preferably developed using the ANSI 92SQL standard, which is usable by either of the RDB's.

[0192] Thin Client Technology

[0193] An object of the application program of the present invention isthat it should operate on a variety of devices, including mobiletelecommunications devices such as cellular phones, handheldweb-browsers, palm pilots, personal digital assistants and other devicesthat can communicate by protocols such as wireless application protocol(WAP).

[0194] Accordingly, because most handheld or mobile devices do not havethe same amount of local storage and processing power as desktopcomputers and set-top boxes, a special version of the applicationprogram of the present invention is envisaged for mobile devices. Thespecial version embodies so-called ‘thin client’ technology in which alot of the operations are performed by a server instead of the deviceitself.

[0195] An “n-tier” architecture is one that is designed generically fora multitude of platforms, for example PC's, PDA's WAP phones and UNIXsystems. Accordingly, in order to maximize the use of mobile devices,the application program of the present invention employs an n-tierarchitecture allowing the majority of the processing to be carried outon the server with the results being sent to the device for rendering.This model allows for a reduced size system to be stored on mobiledevices, with the system logic residing on remote servers. Preferablythe server, which usually has a much wider bandwidth connection thandoes a given client, carries out the role of monitoring datastreams forupdates, prior to transmitting refreshed content to the client.

[0196] The features that are moved from client to server will bedependent upon the device in question. For example, it is possible toprovide the client with more features on a personal digital assistantsuch as a Palm pilot than on a WAP Phone.

[0197] When using thin client technology, it becomes especiallyimportant for the refresh rates of the tiles to be sensibly allocatedaccording to a priority scheme. Accordingly the thin-clientimplementation preferably allows for the automatic allocation of tilepriorities based on the type of information content being transmitted.The thin client implementation also preferably permits the allocation oftile priorities based on recognition of an identifier such as a “qualityof service tag” associated with a source of information.

[0198] In order to provide a level of consistency between the devicesand the servers, the markup language XML is preferably used to wrap thedata that is being transmitted. As previously described, the system ofthe present invention uses a metabase to store information about theuser's current grid and tile configurations. A synchronization procedureallows the metabase to be stored on a server and queried by any device.In this way it is possible to provide consistent grid and tileimplementations independent of the device and its location. Such anembodiment would permit a variant of “peer to peer” communication.

[0199]FIG. 27 provides an example of the way in which wireless devicescan interact with a server. A personal digital assistant (PDA) 2700 or aWAP phone 2702 communicates in a wireless manner with a dial-in bank2704. The dial-in bank communicates data to a server farm 2706 which isconnected to the internet 2710, optionally through a firewall 2708.Alternatively, another personal digital assistant such as 2714 cancommunicate with an internet service provider such as 2712, alsoconnected to the internet 2710. The server farm 2706 is able to providecontent directly to a PDA such as 2700 or indirectly, over the internet,to a PDA such as 2714.

What is claimed is:
 1. A method executed by a processor under thecontrol of a program, said method comprising: partitioning a displayinto an array of tiles wherein information from a first source isdisplayed on a first tile in said array of tiles and information from asecond source is displayed on a second tile in said array of tiles;assigning a first refresh rate to said first tile and a second refreshrate to said second tile, wherein information from at least one of saidfirst source and said second source contains an identifier that is usedin said assigning; updating information from said first source presentedto said first tile in accordance with said first refresh rate; andupdating information from said second source presented to said secondtile in accordance with said second refresh rate.
 2. The method of claim1 wherein said partitioning includes arranging said array of tiles inaccordance with a user-defined array size.
 3. The method of claim 1wherein said partitioning includes arranging said array of tiles into anon-overlapping configuration wherein each tile of said array of tilesis a uniform size and shape
 4. The method of claim 1 wherein saidassigning said first refresh rate is in accordance with a first priorityvalue of a first information source associated with said first tile andsaid assigning said second refresh rate is in accordance with a secondpriority value of a second information source associated with saidsecond tile.
 5. The method of claim 4 additionally comprising:attributing each tile in said array of tiles to a level in a hierarchyof levels according to a priority value associated with each tile. 6.The method of claim 1 wherein said identifier is a quality of servicetag.
 7. The method of claim 1 wherein said first refresh rate isdifferent from said second refresh rate.
 8. The method of claim 1additionally comprising, if information from said first source does notcontain an identifier, assigning said first refresh rate in accordancewith a user-specified refresh rate.
 9. A method executed by a processorunder the control of a program, said method comprising: partitioning adisplay into an array of tiles wherein information from a first sourceis displayed on a first tile in said array of tiles and information froma second source is displayed on a second tile in said array of tiles;assigning a first refresh rate to said first tile and a second refreshrate to said second tile, based on a first type of said information fromsaid first source and a second type of said information from said secondsource; updating information from said first source presented to saidfirst tile in accordance with said first refresh rate; and updatinginformation from said second source presented to said second tile inaccordance with said second refresh rate.
 10. The method of claim 9wherein said first type and said second type are different from oneanother.
 11. An electronic readable memory to direct a device,comprising: a first set of instructions to control simultaneouscommunication with a plurality of information sources; a second set ofinstructions to partition a display unit of the device into an array oftiles; a third set of instructions to associate a first informationsource of said plurality of information sources with a first tile ofsaid array of tiles and a second information source of said plurality ofinformation sources with a second tile of said array of tiles, such thatinformation from said first source is displayed on said first tile andinformation from said second source is displayed on said second tile,and wherein information from at least one of said first source and saidsecond source contains an identifier; a fourth set of instructions toretrieve information from said first information source in accordancewith a first retrieval rate and to retrieve information from said secondinformation source in accordance with a second retrieval rate wherein atleast one of said first and second retrieval rates is allocated basedupon said identifier; and a fifth set of instructions to presentinformation to said first tile in accordance with said first retrievalrate and present information to said second tile in accordance with saidsecond retrieval rate.
 12. The electronic readable memory of claim 11further comprising a set of instructions to process a network datastreamfrom a network data source.
 13. The electronic readable memory of claim11 further comprising a set of instructions to process a tuner signalfrom a tuner device.
 14. The electronic readable memory of claim 11wherein said second set of instructions arrange said array of tiles on adisplay unit of a device selected from the group consisting of: cellularor mobile telephone, television, set top box, desktop computer, laptopcomputer, notebook computer, digital audio player, games player, digitalradio, navigation system, and personal digital assistant.
 15. Theelectronic readable memory of claim 11 wherein said second set ofinstructions produce an array of non-overlapping tiles wherein each tilehas a uniform size and shape.
 16. The electronic readable memory ofclaim 11 wherein said fourth set of instructions assign said firstretrieval rate and said second retrieval rate in accordance with apredetermined priority scheme.
 17. The electronic readable memory ofclaim 11 further comprising a set of instructions to selectively assigna selected or unselected state to specified tiles of said array oftiles.
 18. The electronic readable memory of claim 11 further comprisinga set of instructions to deliver selected textual content from saidfirst information source to said first tile.
 19. The electronic readablememory of claim 11 further comprising a set of instructions to deliver avideo signal to a selected tile of said array of tiles.
 20. Theelectronic readable memory of claim 11 further comprising a set ofinstructions to deliver a frame of a broadcast TV signal to a selectedtile of said array of tiles.
 21. The electronic readable memory of claim11 further comprising a set of instruction to deliver a web page to aselected tile of said array of tiles.
 22. The electronic readable memoryof claim 11 wherein said first retrieval rate is different from saidsecond retrieval rate.
 23. A system for facilitating the organizationand management of multiple data sources, comprising: a device thatincludes a processor configured to execute instructions, a memoryconnected to said processor to store at least one program that includesa graphical user interface, and an input device, wherein said processorexecutes instructions to: control simultaneous communication with aplurality of information sources; partition a display into an array oftiles; associate a first information source of said plurality ofinformation sources with a first tile of said array of tiles and asecond information source of said plurality of information sources witha second tile of said array of tiles, such that information from saidfirst information source is displayed on said first tile and informationfrom said second information source is displayed on said second tile,wherein information from at least one of said first source and saidsecond source contains an identifier; retrieve information from saidfirst information source in accordance with a first retrieval rate andretrieve information from said second information source in accordancewith a second retrieval rate wherein said first and second retrievalrates are allocated based upon said identifier; and present informationto said first tile in accordance with said first retrieval rate andpresent information to said second tile in accordance with said secondretrieval rate.